Papers with multi-hop inference
TextGraphs 2019 Shared Task on Multi-Hop Inference for Explanation Regeneration (D19-53)
Copied to clipboard
| Challenge: | Detailed extended analyses of all submitted systems showed large relative improvements in accessing the most challenging multi-hop inference problems, while absolute performance remains low. |
| Approach: | The Shared Task on Multi-Hop Inference for Explanation Regeneration asks participants to regenerate detailed gold explanations for elementary science questions by selecting facts from a knowledge base of semi-structured tables. |
| Outcome: | The top-performing system achieved a mean average precision of 0.56 . the task combines facts from a knowledge base and supervised training data . |
Fine-tuning BERT with Focus Words for Explanation Regeneration (2020.starsem-1)
Copied to clipboard
| Challenge: | Existing approaches to explain the correct answer in multiple-choice QA are low in F-scores and lack of performance. |
| Approach: | They introduce a lightweight focus feature in a transformer-based NLP model and examine performance improvements. |
| Outcome: | The proposed model achieves the highest scores, second only to a computationally intensive system. |
When Thoughts Meet Facts: Reusable Reasoning for Long-Context LMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent Long-Context Language Models (LCLMs) do not capture how evidence should be connected . a new framework that integrates thought templates into LCLM frameworks is proving useful . |
| Approach: | They propose a framework that iteratively refines reusable reasoning patterns derived from prior problem solving to improve their templates. |
| Outcome: | The proposed framework outperforms baselines on knowledge-intensive multi-hop reasoning benchmarks and practical scenarios without retrieval. |
Case-Based Abductive Natural Language Inference (2022.coling-1)
Copied to clipboard
| Challenge: | Recent approaches for multi-hop inference construct explanations considering each test case in isolation, but semantic drift causes wrong conclusions. |
| Approach: | They propose an abductive framework for multi-hop NLI exploring the retrieve-reuse-refine paradigm in Case-Based Reasoning. |
| Outcome: | The proposed model can be integrated with sparse and dense pre-trained encoders to improve multi-hop inference, or adopted as an evidence retriever for Transformers. |
Can LLMs reason over extended multilingual contexts? Towards long-context evaluation beyond retrieval over haystacks (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing multilingual long-context benchmarks are myopic and inherently limited, as successful recall alone does not indicate a model’s capacity to reason over extended contexts. |
| Approach: | They propose a new synthetic benchmark for multilingual long-context reasoning that includes bAbI-style tasks that test multi-hop inference, aggregation, and epistemic reasoning. |
| Outcome: | The proposed benchmarks are based on a multilingual long-context model and span seven languages. |
KQA Pro: A Dataset with Explicit Compositional Programs for Complex Question Answering over Knowledge Base (2022.acl-long)
Copied to clipboard
Shulin Cao, Jiaxin Shi, Liangming Pan, Lunyiu Nie, Yutong Xiang, Lei Hou, Juanzi Li, Bin He, Hanwang Zhang
| Challenge: | Existing benchmarks for Complex KBQA lack compositional reasoning capabilities . Existing methods for Complex questions are poor in diversity or scale . |
| Approach: | They propose a compositional programming language to represent the reasoning process of complex questions. |
| Outcome: | The proposed dataset includes around 120K diverse natural language questions . it provides a compositional and interpretable programming language to represent the reasoning process of complex questions based on the proposed model . |
MultiHoax: A Dataset of Multi-hop False-premise questions (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks focus on single-hop FPQs, but real-world reasoning often requires multi-hop inference . state-of-the-art LLMs struggle to detect false premises across different countries, knowledge categories, and multi-step reasoning types. |
| Approach: | They propose a benchmark to evaluate Large Language Models' ability to handle false premises in complex, multi-step reasoning tasks. |
| Outcome: | The proposed tests show that state-of-the-art LLMs struggle to detect false premises across different countries, knowledge categories, and multi-hop reasoning types. |
Autoregressive Reasoning over Chains of Facts with Transformers (2020.coling-main)
Copied to clipboard
| Challenge: | Existing work on this task either evaluates facts in isolation or artificially limits the possible chains of facts, thus limiting multi-hop inference. |
| Approach: | They propose an iterative inference algorithm that decomposes the selection of facts from a corpus autoregressively and conditioning the next iteration on previously selected facts. |
| Outcome: | The proposed method outperforms the previous state-of-the-art in terms of precision, training time and inference efficiency on the WorldTree dataset. |
WorldTree V2: A Corpus of Science-Domain Structured Explanations and Inference Patterns supporting Multi-Hop Inference (2020.lrec-1)
Copied to clipboard
Zhengnan Xie, Sebastian Thiem, Jaycie Martin, Elizabeth Wainwright, Steven Marmorstein, Peter Jansen
| Challenge: | Standardized science questions require combining an average of 6 facts and as many as 16 facts to answer and explain. |
| Approach: | They propose to combine an average of 6 facts and as many as 16 facts to produce an answer for complex questions. |
| Outcome: | The proposed model is based on a corpus of 5,114 standardized science exam questions . it uses multi-fact explanations that combine science knowledge and world knowledge . |
Reinforced Query Reasoners for Reasoning-intensive Retrieval Tasks (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for retrieval of information excel at textual and semantic matching but struggle in reasoning-intensive retrieval tasks. |
| Approach: | They propose a family of small-scale language models for query reasoning and rewriting in reasoning-intensive retrieval. |
| Outcome: | The proposed model outperforms existing models on a BRIGHT benchmark with BM25 retrievers. |